SYSTEMS AND METHODS FOR BOUNDARY DETECTION IN IMAGES 



BACKGROUND OF THE INVENTION 

1. Field of Invention 

[0001] This invention relates to boundary detection and boundary location 
determination between two regions in images. 

2. Description of Related Art 

[0002] Many conventional machine visions systems used in locating the edges of 
features in images are based primarily or exclusively on applying gradient operations to the 
intensity values of the original image pixels. In applying gradient operations, these systems 
perform edge-location using the contrast inherent in the original intensity of an image. This 
operation is often used for machine visions systems that emphasize determining the location 
of edges in images of man-made work pieces with a high degree of precision and reliability. 
In these cases, the geometry of the edges is often well-behaved and predictable, thus 
providing constraints that can be applied to the edge location operations so that good results 
may be obtained for the majority of these images. It is also well known to use filters prior to 
edge detection operations to improve the reliability of the intensity gradient-type operations in 
finding points along an edge, and to exclude outliers from the located edges points after edge 
detection to further increase the reliability of the detected edge location. 

[0003] There are several conventional vision machines that use these methods. 
These vision machines also typically include software that provides one or more "edge tools." 
The edge tools are special cursors and/or graphical user interface (GUI) elements that allow 
an operator of a machine vision system to more easily input useful information and/or 
constraints used with the underlying edge-location method. 

[0004] However, as is well known in the field of image processing, these 
conventional methods can become unreliable when the image regions near edges exhibit a 
high degree of texture or when the edge is defined by a change in texture, color, or other 
image characteristics that do not always correspond to well-behaved intensity gradients in the 
image. The images associated with textured edges are inherently irregular or noisy because 
each texture region near a particular edge is imaged as a high spatial frequency intensity 
variation near the edge. Thus, the intensity gradient-type operations previously discussed 
tend to return noisy results, which subsequently result in poor detection of the edge locations. 
Although filtering operations can be used to reduce the noise in these situations, the filtering 



operations can also unintentionally further disturb the image in a way that distorts the 
detected edge location. Furthermore, in some cases, for example when the average intensities 
in the texture regions bordering the edge are approximately the same, intensity gradient 
operations become completely unreliable for finding the location of the edges. Thus, in such 
situations, the conventional methods cannot precisely detect an edge location of an image 
because there is no significant intensity gradient or differential that can be clearly detected. 

[0005] In images containing multiple distinct objects or regions having various 
textures, a wide variety of texture-based image-segmentation methods are known. For 
example, one method can group or classify image pixels into local regions based on the 
values of particular texture metrics. Such methods define a border which separates the pixels 
grouped or classified in one region from the pixels grouped or classified in the other region, 
as a by-product of classification process. However, such methods are typically designed and 
applied for object recognition, object tracking and the like. 

[0006] A common problem associated with these existing image segmentation 
systems is the rigidity of the system structure. Systems which include a great variety of 
texture filters for robustness are too slow to support high-speed industrial throughput 
requirements. Systems which limit the number of texture filters and or use a limited number 
of predetermined parameters usable as thresholds in detecting region membership are often 
unreliable when applied to a wide variety of textures. Thus, such existing segmentation 
systems are insufficiently versatile, robust and/or fast for use in a general-purpose commercial 
machine vision system. 

[0007] Furthermore, such segmentation methods have not been well-developed for 
finding relatively precise positions for edge locations at the boundaries between regions. It is 
generally recognized that accurate edge/boundary preservation is a goal that conflicts to some 
extent with operations, such as energy estimation, which are essential for accurate pixel 
grouping or classification. For example, U.S. Patent 6,178,260 to Li et al. discloses a method 
used for character recognition, where a local roughness and a peak-valley count is determined 
for a window and/or subwindow of an image. The input image data for the window is 
subsequently classified based on the local roughness and the peak-valley count. This method 
may be complemented by using a pattern-detecting edge class that tries to identify line art or 
kanji regions that could otherwise be missed by the roughness and peak-valley classification. 
This image segmentation method is more robust than many previous methods, and adapts to a 
current image. However, this method does not disclose any specific methods or tools of 



particular use for locating the position of a boundary between the classification regions with 
robustness and precision. 

[0008] U.S. Patent 6,1 1 1,983 to Fenster et al. discloses a method used for shape 
recognition that can be used with medical images. In this method, a shape model is "trained" 
for parameter settings in an objective function, based on training data for which the correct 
shape is specified. This training can be advantageously applied to models in which a shape or 
a boundary is treated in a sectored fashion, with training individually applied to each sector. 
The sectors may be characterized by a variety or combination of features, and the features are 
adjusted to generate a desirable sector dependent objective function. This method is more 
robust than many previous methods, and adapts to a current image. However, the method 
does not disclose any specific methods or tools for locating the position of a boundary 
between various sectors with robustness and precision. 

[0009] For application to general purpose commercial machine vision systems, it is 
also highly desirable or necessary that the various image processing methods incorporated 
into the system can be set up and operated for particular images by relatively unskilled users, 
that is, users who are not skilled in the field of image processing. Thus, it is a particular 
problem to create a machine vision system which locates textured edges in a versatile, robust, 
fast and relatively precise way, while at the same time adapting and governing that machine 
vision system edge detection process through the use of a simple user interface that is 
operable by a relatively unskilled operator. 

SUMMARY OF THE INVENTION 
[0010] Accordingly, texture-based segmentation methods and image-specific 
texture-based segmentation methods have not been well-developed for finding relatively 
precise positions for edge locations at the boundaries between regions. Furthermore, such 
methods have not been combined with a method that automatically streamlines them and 
subordinates them to other edge or boundary detection operations according to the reasonably 
well-behaved and predictable characteristics of particular edges found on industrial inspection 
objects. Moreover, these methods have not been supported by a simple user interface or 
compatible "edge tools" which can be used by operators having little or no understanding of 
the underlying mathematical or image processing operations. 

[0011] Finally, no conventional machine vision system user interface supports both 
the operation of conventional intensity gradient-type edge locating operations and 



texture-type edge-locating operations with substantially similar edge-tools and/or related 
GUIs, or combines both types of operations for use with a single edge tool. 

[0012] Accordingly, because many operators of conventional machine vision 
systems desire a more standardized edge locating capability which supports increasingly 
robust operations with minimal user understanding and/or intervention, there is a need for 
systems and methods that can be used with existing machine vision systems that can precisely 
detect the position of a boundary, i.e., an edge, between regions using image characteristics 
other than intensity gradients or differentials so that images of edges that are not well-defined 
by changes in intensity can be more accurately detected and located. 

[0013] This invention provides systems and methods that accurately locate an edge 
position based on a number of different characteristics of the image. 

[0014] This invention separately provides systems and methods that accurately 
locate an edge position bounded or defined by one or two significantly textured regions as an 
easily integrated supplement and/or alternative to intensity-gradient type edge locating 
operations. 

[0015] This invention separately provides systems and methods that accurately 
locate an edge position bounded by one or two significantly colored regions or color-textured 
regions as an easily integrated supplement and/or alternative to intensity-gradient type edge 
locating operations. 

[0016] This invention separately provides systems and methods where the decisions 
and operations associated with locating an edge position can be performed manually with the 
aid of the GUI, semi-automatically or automatically. 

[0017] This invention separately provides systems and methods that accurately 
locate an edge position bounded by one or two highly textured regions using 
adaptively-selected texture filters and/or texture features. 

[0018] This invention separately provides systems and methods that define a 
plurality of specific training regions-of-interest in the vicinity of an edge-location operation, 
where the specific training regions are used to determine a texture-discriminating filter and/or 
feature set which best supports edge-location operations at an edge or boundary between the 
training regions. 

[0019] This invention separately provides systems and methods that determine a 
customized case-specific edge-finding routine that operates with particular speed and 
reliability when finding similar case-specific edges in images of similar imaged parts. 



[0020] This invention separately provides systems and methods where certain 
decisions and operations associated with the determination of a customized case-specific 
edge-finding routine can be performed manually with the aid of the GUI, semi-automatically 
or automatically. 

[0021] In various exemplary embodiments of the systems and methods according to 
this invention, a user can invoke a boundary detection tool, alternatively called an edge tool, 
to perform a texture-based edge-finding operation, possibly along with a conventional 
intensity gradient edge-locating operation, to define a primary area of interest that will include 
an edge to be located within a captured image of an object. The boundary detection tool in 
accordance with the systems and methods according to this invention is useable to locate 
edges in a current object, and to locate corresponding edges of similar objects in the future. 

[0022] A boundary detection tool in accordance with the systems and methods 
according to this invention optionally allows a user to specify the shape, the location, the 
orientation, the size and/or the separation of two or more pairs of sub-regions-of-interest 
bounding the edge to be located. Alternatively, the machine vision systems and methods 
according to this invention can operate automatically to determine the sub-regions-of-interest. 
If conventional intensity gradient-based edge-locating operations are not appropriate for 
locating the edge included in the primary region-of-interest, then the sub-regions-of-interest 
are used as training regions to determine a set of texture-based features which can be used to 
effectively separate the feature values of pixels on either side of the included edge into two 
distinct classes or clusters. A pseudo-image, such as a membership image, is calculated using 
the feature images. Gradient operations can then be applied to the membership image to 
detect the desired edge and determine its location. Post-processing can be applied to the edge 
data, using input data related to known features and approximate locations of the edge, to 
remove outliers and otherwise improve the reliability of the edge location. These and other 
features and advantages of the this invention allow relatively unskilled users to operate a 
general-purpose machine vision system in a manner that precisely and repeatably locates 
edges in a variety of situations where conventional intensity gradient methods locate edges 
unreliably or fail to locate the edges altogether. 

[0023] These and other features and advantages of this invention are described in, or 
are apparent from, the following detailed description of various exemplary embodiments of 
the systems and methods according to this invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 
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[0024] Various exemplary embodiments of this invention will be described in detail, 
with reference to the following figures, wherein: 

Fig. 1 is an exemplary block diagram of a vision system usable with the edge 
detection systems and methods according to this invention; 
5 Fig. 2 illustrates a detailed exemplary embodiment of the various circuits or routines 

of Fig. 1 usable with the edge detection systems and methods according to this invention; 

Fig. 3 illustrates two images of exemplary objects having two significantly textured 
regions and a boundary that can be detected and located using the edge tools and edge 
detection systems and methods according to this invention; 
10 Fig. 4 illustrates exemplary regions-of-interest generated by and usable with the 

systems and methods according to this invention; 
q Fig. 5 illustrates an image of one exemplary embodiment of a pseudo-image with a 

scan line used with various embodiments of the systems and methods according to this 
H invention; 

1%j Fig. 6 illustrates an image of one exemplary embodiment of multiple edge locations 

: : detected using the edge detection systems and methods according to this invention; 

Fig. 7 is a flowchart outlining one exemplary embodiment of a method for 
N= determining edge locations in an image according to this invention; 

Fig. 8 is a flowchart outlining in greater detail one exemplary embodiment of the 
2CP method for determining an area-of-interest of Fig. 7 according to this invention; 

Fig. 9 is a flowchart outlining in greater detail one exemplary embodiment of the 
method for determining feature images of Fig. 7 according to this invention; 

Fig. 10 is a flowchart outlining in greater detail one exemplary embodiment of the 
method for performing feature selection of Fig. 7 according to this invention; 
25 Fig. 1 1 is a flowchart outlining an exemplary embodiment of a method for 

determining a pseudo-image of Fig. 7 according to this invention; 

Fig. 12 is a flowchart outlining an exemplary embodiment of a method for detecting 
and selecting edge point locations of Fig. 7 according to this invention; 

Fig. 13 is a flowchart outlining in greater detail one exemplary embodiment of the 
30 method for selecting a representative pair of regions-of-interest of Fig. 10 according to this 
invention; 

Fig. 14 is a flowchart outlining an exemplary embodiment of a method for selecting 
valid edge point locations of Fig. 12 according to this invention; 
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Fig. 15 is a flowchart outlining one exemplary embodiment of a method for using a 
tool defined according to the method outlined in Figs. 7-14 to identify edges in a second 
image according to this invention; and 

Fig. 16 is a flowchart outlining in greater detail one exemplary embodiment of the 
method for selecting valid edge point locations of Fig. 14 according to this invention. 
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
[0025] The systems and methods of this invention can be used in conjunction with 
the machine vision systems and/or the lighting calibration systems and methods disclosed in 
U.S. Patent 6,239,554 Bl, which is incorporated herein by reference in its entirety. 

[0026] With regard to the terms "boundaries" and "edges" as used herein, the terms 
"boundaries" and "edges" are generally used interchangeably with respect to the scope and 
operations of the systems and methods of this invention. However, when the context clearly 
dictates, the term "edge" may further imply the edge at a discontinuity between different 
surface planes on an object and/or the image of that object. Similarly, the term "boundary" 
may further imply the boundary at a discontinuity between two textures, two colors, or two 
other relatively homogeneous surface properties, on a relatively planar surface of an object, 
and/or the image of that object. 

[0027] For simplicity and clarification, the operating principles and design factors of 
this invention are explained with reference to one exemplary embodiment of a vision system 
according to this invention, as shown in Fig. 1. The basic explanation of the operation of the 
vision system shown in Fig. 1 is applicable for the understanding and design of any vision 
system that incorporates the boundary detection systems and methods according to this 
invention. 

[0028] Fig. 1 shows one exemplary embodiment of a vision system 10 incorporating 
one exemplary embodiment of the boundary detection systems and methods according to this 
invention. As shown in Fig. 1, the vision system 10 includes a control portion 100 and a 
vision system components portion 200. The vision system components portion 200 includes 
a stage 210 having a central transparent portion 212. An object 20 to be imaged using the 
vision system 10 is placed on the stage 210. Light emitted by one or more of the light sources 
220-240 illuminates the object 20. The light from the one or more light sources 220-240 
passes through a lens system 250 after illuminating the part 20, and possibly before 
illuminating the object 20, and is gathered by a camera system 260 to generate an image of 
the object 20. The image of the part 20 captured by the camera system 260 is output on a 



signal line 262 to the control portion 100. The light sources 220-240 used to illuminate the 
object 20 include a stage light 220, a coaxial light 230, and a surface light 240, such as a ring 
light or a programmable ring light, all connected to the control portion 100 through 
connecting lines or buses 221 , 23 1 and 241 , respectively. 

[0029] The distance between the stage 210 and the camera system 260 can be 
adjusted to change the focus of the image of the object 20 captured by the camera system 260. 
In particular, in various exemplary embodiments of the vision system 10, the position of the 
camera system 260 along a vertical axis is changeable relative to a fixed stage 210. In other 
various exemplary embodiments of the vision system 10, the position of the stage 210 along 
the vertical axis can be changed relative to a fixed camera system 260. In further various 
exemplary embodiments of the vision system 10, the vertical positions of both the camera 
system 260 and the stage 210 can be altered to maximize the focus range of the vision system 
10. 

[0030] As shown in Fig. 1, one exemplary embodiment of the control portion 100 
includes an input/output interface 1 10, a controller 120, a memory 130, an area of interest 
generator 150, and a power supply 190 including an illumination power supply portion 191, 
each interconnected either by a data/control bus 140 or by direct connections between the 
various elements. The memory 130 includes a video tool memory portion 131, a filter 
memory portion 132, and a part program memory portion 133, each also interconnected by 
the data/control bus 140 or by direct connections. The connecting lines or buses 221, 231 and 
241 of the stage light 220, the coaxial light 230, and the surface light 240, respectively, are all 
connected to the illumination power supply portion 191. The signal line 262 from the camera 
system 260 is connected to the input/output interface 1 10. Also, a display 102 can be 
connected to the input/output interface 110 over a signal line 103. One or more input devices 
1 04 can be connected over one or more signal lines 105. The display 102 and the one or more 
input devices 104 can be used to view, create and/or modify part programs, to view the 
images captured by the camera system 260 and/or to directly control the vision system 
components 200. However, it should be appreciated that, in a fully automated system having 
a predefined part program, the display 102 and/or the one or more input devices 104, and the 
corresponding signal lines 103 and/or 105, may be omitted. 

[0031] As shown in Fig. 1, the vision system 10 also includes a filtered image 
analyzing circuit or routine 310, a case-specific filter selection circuit or routine 350, a 
pseudo-image generating circuit or routine 360, an edge point analyzing circuit or routine 



370, a boundary locating and refining circuit or routine 380, and an optional edge mode 
determining circuit or routine 390, each also interconnected by the data/control bus 140 or by 
direct connections. 

[0032] The memory portion 130 stores data usable to operate the vision system 
components 200 to capture an image of the object 20 such that the input image of the object 
20 has desired image characteristics. The memory portion 130 further stores data usable to 
operate the vision system to perform various inspection and measurement operations on the 
captured images, manually or automatically, and to output the results through the input/output 
interface 110. The memory 130 also contains data defining a graphical user interface 
operable through the input/output interface 110. 

[0033] The video tool memory portion 131 includes data defining various video 
tools usable with the graphical user interface, and in particular, one or more edge or boundary 
tools usable with the area of interest generator 1 50 to define and store in memory data 
associated with an edge locating operation in an area of interest within the captured image. 
An exemplary edge/boundary detection tool and the associated data are described in greater 
detail below with reference to Figs. 3 and 4. The filter memory portion 132 includes data 
defining various image filtering operations usable with the systems and methods according to 
this invention, as described in detail further below. The part program memory portion 133 
includes data defining various operations usable to create and store a sequence of operations 
or routines for subsequent automatic operation of the vision system 10. 

[0034] The filtered image analyzing circuit or routine 310 applies various candidate 
filters to modify and/or analyze a textured input image in a current area of interest, and 
determines filtered image results based on the modifications and/or analysis. The filtered 
image results are usable to determine which of the candidate filters best emphasize or isolate 
the location of an edge in the area of interest. The case-specific filter selection circuit or 
routine 350 selects the case-specific filters that best emphasize or isolate the location of the 
edge in the area of interest, based on the various filtered image results. The case-specific 
filter selection circuit or routine 350 may also record the case-specific filter selection in one 
or more portions of the memory 130. 

[0035] The pseudo-image generating circuit or routine 360 generates a pseudo- 
image in the area of interest based on the selected case-specific filters. The pseudo-image 
emphasizes or isolates the location of the edge relative to the obscured characteristics of the 
textured edge in the input image. The edge point analyzing circuit or routine 370 is then 
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applied to the pseudo-image in the area of interest, to estimate one or more edge points in the 
pseudo-image. The edge point analyzing circuit or routine 370 may also perform operations 
to refine an initial edge point estimate, based on additional information. The edge point 
analyzing circuit or routine 370 may also record one or more edge detection parameters 
associated with the estimated edge points in one or more portions of the memory 130. 

[0036] The boundary locating and refining circuit or routine 380 analyzes a plurality 
of estimated edge points to determine if they correspond to criteria for a reliable edge. The 
boundary locating and refining circuit or routine 380 also governs the refinement or 
elimination of spurious edge points and finally determines overall edge location data based on 
reliable edge points. The boundary locating and refining circuit or routine 380 may also 
record the edge location data in one or more portions of the memory 130 or output it through 
the input/output interface 110. 

[0037] The edge mode determining circuit or routine 390 can be an optional element 
of the control system portion 100. It should be appreciated that the control system portion 
100 also includes known circuits or routines to perform known edge detection operations on 
input images acquired by the vision system 100. Such known edge-detection known circuits 
or routines may be included in the edge point analyzing circuit or routine 370 and/or the 
boundary locating and refining circuit or routine 380, for example. Depending on the scope 
of operation of various elements such as the edge tools in the video memory portion 131, the 
area of interest generator 150, the edge point analyzing circuit or routine 370 and the 
boundary locating and refining circuit or routine 380, such elements may operate to 
independently determine whether a given area of interest is appropriately analyzed by an edge 
detection applied to the input image or an edge detection applied to a pseudo-image. 
However, when such elements cannot independently determine whether a given area of 
interest is appropriately analyzed by edge detection applied to the input image or edge 
detection applied to a pseudo-image, the edge mode determining circuit or routine 390 can be 
included to determine the appropriate mode of operation for the various other elements 
performing the edge detection operations. 

[0038] Fig. 2 illustrates a detailed exemplary embodiment of various circuits or 
routines of the vision system 10 described above with respect to Fig. 1 . As shown in Fig. 2, 
the filtered image analyzing circuit or routine 310 includes a candidate filter selection circuit 
or routine 3 1 1 , a feature image generating circuit or routine 3 12, a regions-of-interest 
generating circuit or routine 313, and a regions-of-interest comparing circuit or routine 314, 
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each interconnected by the data/control bus 140 or by direct connections. The edge point 
analyzing circuit or routine 370 includes a scan line determining circuit or routine 377, a edge 
point detection circuit or routine 378, and an edge point refining circuit or routine 379, each 
also interconnected by the data/control bus 140 or by direct connections. The boundary 
locating and refining circuit or routine 380 includes a shape analysis circuit or routine 381, an 
outlier elimination circuit 382, and a location determining circuit 383, each also 
interconnected by the data/control bus 140 or by direct connections. The edge mode 
determining circuit or routine 390 includes an edge tool interpreting circuit or routine 391 and 
an area of interest analyzing circuit or routine 392, each also interconnected by the 
data/control bus 140 or by direct connections. 

[0039] In various exemplary embodiments of the filtered image analyzing circuit or 
routine 310, the elements 311-314 operate as follows: 

[0040] The candidate filter selection circuit or routine 3 1 1 selects the set of 
candidate filters that will be applied to the input image to obtain feature images or the like 
corresponding to the candidate filters. The candidate filters are selected from the set of filters 
included in the filter memory portion 132, which in one exemplary embodiment includes one 
or more predetermined groups of candidate filters. Each such group includes filters that are 
associated with enhancing edge detection and location for images that exhibit a particular set 
of characteristics around their edges to be detected. The candidate filter selection circuit or 
routine 311 selects particular candidate filters depending on the characteristics of the input 
image. Such characteristics may include, for example, whether there is significant texture on 
one or both sides of the edge to be detected, whether the image is a gray level image or a 
color image, and the like, as described further below. For various images, the candidate filter 
selection circuit or routine 311 may select all the filters in the filter memory portion 132. In 
various exemplary embodiments, the candidate filter selection circuit or routine 311 
automatically selects the candidate filters and in other exemplary embodiments the selection 
is based on user input. 

[0041] In various exemplary embodiments, the predetermined subsets of candidate 
filters selectable by the candidate filter selection circuit or routine 311 include: a subset 
including filters that establish one or more a feature images based on the gradient of a Sobel 
operator, a subset including filters that establish one or more feature images based on Law's 
filters, i.e., a set of 25 filters incorporating 5x5 (or optionally, 3x3) pixel masks or windows, 
and a subset including filters that establish one or more feature images based on Gabor's 



12 

filters. The inventor has used the Sobel gradient filter with success when the edge to be 
detected includes significant texture on one side of the edge and insignificant texture on the 
other side of the edge. The inventor has used Law's filters with success when the edge to be 
detected includes significant and fine textures on both sides of the edge. The inventor has 
used Gabor's filters with success when the edge to be detected includes significant and fine 
textures and/or directional features on both sides of the edge. Also, to detect the boundary 
between color regions in color images the inventor has used moving average filters with 
success. 

[0042] These various filter subsets tend to operate with respective short, medium 
and longer execution times. Thus, they are conveniently selected to match the appropriate 
texture conditions in a particular area of interest. In various exemplary embodiments, the 
candidate filter selection circuit or routine 311 includes operations similar to, or interacts 
with, the area of interest analyzing circuit or routine 392 described below, to measure one or 
more texture characteristics in evaluation regions on both sides of the edge in the area of 
interest. The candidate filter selection circuit or routine 311 then compares the resulting 
texture measurements to predetermined criteria associated with the various candidate filter 
groups and selects the appropriate predetermined candidate filter subset. For example, if 
there is a low variance value on one side of the border the previously discussed Sobel-type 
filter can be used. If a directional texture characteristic is detected, Gabor's filters can be 
used. If a fine non-directional texture is detected on both sides of the boundary, Law's filters 
can be used. Color filters can be used for color images, and so on. Methods for 
characterizing various textures are well know to one skilled in the art, and are also discussed 
in the references cited herein. 

[0043] It should be appreciated that any known, or later-developed filter and/or set 
of image filtering steps, can be used in various embodiments of the edge detection systems 
and methods according to this invention. 

[0044] It should also be appreciated that the terms "candidate filter" and "selected 
filter" or "case-specific filter," as used herein in various exemplary embodiments, may 
encompasses all the necessary functions or components necessary to produce a filtered image 
using a particular filter function, a feature image resulting from applying a local energy 
function to a filtered image, a normalized feature image based on the feature image, or the 
like. Also included may be the functions or operations needed to determine any now known 
or later developed metric that is suitable for characterizing any of the preceding types of 
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images. More generally, the terms candidate filter and selected filter encompass not only a 
particular filter function, but any unique functions or components associated with that 
particular filter function, which must be used by the filtered image analyzing circuit 310, 
and/or the feature image generating circuit 312 and/or the regions-of-interest generating 
circuit or routine 313, in various exemplary embodiments, to generate one or more partial 
filtered image results corresponding to that particular filter function. Thus, the terms 
candidate filter and selected filter, as used herein, refer to all the unique elements required to 
determine a corresponding partial filtered image result corresponding to a particular filter 
function, as described below. Because of their scope in various exemplary embodiments, 
filters and groups of filters are also sometimes referred to as filter methods herein. 

[0045] The feature image generating circuit or routine 312, generates at least one 
feature image or the like based on the selected candidate filters. The feature image generating 
circuit 312 is applied to the original input image according to the area of interest generated by 
the area of interest generator 150. In an exemplary embodiment, one feature image Ffc is 
generated for each candidate filter k. A feature image is generated, in general, by filtering the 
input image data with a particular filter function and applying a local energy function to the 
filtered image data. The local energy function, in general, rectifies and smoothes the image 
signals represented in the filtered image data. Exemplary local energy functions included 
summing the magnitudes of the filtered image pixel values in a window surrounding each 
pixel to determine each pixel value of the feature image, and summing the squares of the 
filtered image pixel values in a window surrounding each pixel to determine each pixel value 
of the feature image. 

[0046] Furthermore, in an exemplary embodiment, each feature image can be 
normalized so that the partial filtered image results corresponding to each candidate filter are 
more easily compared, as described further below. In such cases, the normalized feature 
image is then the feature image represented by the symbol herein. Normalization methods 
are well known in the art. For example, the pixel values of each feature image can be 
normalized to a range which has zero mean and unit variance. In general, any appropriate 
known or later-developed normalization method can be used. 

[0047] The regions-of-interest generating circuit or routine 313, allows an 
automated process, or the user, to define various regions-of-interest within the vicinity of the 
area of interest. The regions-of-interest generating circuit or routine 313 also determine 
"partial filtered image results" based on the regions of interest. One partial filtered image 
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result is determined for each region-of-interest in each feature image generated by the 
feature image generating circuit or routine 3 12. Each partial filtered image result in a region- 
of-interest may in various exemplary embodiments be a filtered image, a feature image 
resulting from applying a local energy function to a filtered image, a normalized feature 
image, or the like, or any now known or later developed metric that is suitable for 
characterizing any of the preceding types of images or their known variants. In an exemplary 
embodiment, the partial filtered image result in a region-of-interest is the average value of the 
pixel values of a normalized feature image in that region of interest. A "partial" filtered 
image results should be understood as an "intermediate" result, which may be used to 
determine one or more "final" filtered image results, also called simply "filtered image 
results" herein, for a filtered image or feature image. A filtered image result for a filtered 
image or feature image generally indicates the ability of that image to emphasize or isolate a 
boundary to be detected according to the systems and methods described herein. 

[0048] The regions-of-interest generating circuit or routine 313 generates the 
regions-of-interest based on the data associated with an appropriately located edge tool and/or 
an operation of the area of interest generator 150. The regions-of-interest are the same or 
congruent for each feature image F^. In one exemplary embodiment, the regions-of-interest 
are defined in one or more pairs symmetrically located about a central point approximately on 
the edge to be located in the area of interest. The central point may be the point P0 described 
further below. More generally, the regions-of-interest include at least one pair of regions that 
lie on opposite sides of the boundary in the area of interest. A region-of-interest should be 
large enough to capture all typical texture characteristics exhibited on one side of the 
boundary and relatively near the boundary in the area of interest. Generating multiple 
regions-of-interest surrounding the boundary and/or the central point on the boundary in the 
area of interest has two advantages. Firstly, if there are texture anomalies such a scratches or 
dirt in the area of interest, some of the regions-of-interest should be free of the anomalies. 
Secondly, multiple regions can be generated automatically in a generic fashion, with a very 
good chance that the regions-of-interest comparing circuit or routine 314 will find a good 
representative pair of the regions of interest, as describe below. Exemplary regions of interest 
are also shown and discussed in regard to Fig. 4, below. 

[0049] The regions-of-interest comparing circuit or routine 314 compares the partial 
filtered image results previously determined in the various regions-of-interest to pick the 
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representative regions-of-interest pair that best reflect the texture differences on each side of 
the boundary. In one exemplary embodiment, the regions-of-interest comparing circuit or 
routine 314 determines the difference between a feature image metric determined in each 
symmetrically located pair of regions-of-interest by the regions-of-interest generating circuit 
or routine 313. The difference is determined for each region-of-interest pair in each feature 
image The feature image metric may be the previously described average value of the 
pixel values of a normalized feature image Ffc in each region-of-interest, for example. The 
regions-of-interest comparing circuit or routine 314 then picks the regions-of-interest pair that 
exhibits the greatest difference as the representative regions-of-interest (RROIi and RROI2), 
that best reflect the texture differences on each side of the boundary. 

[0050] In another exemplary embodiment, the regions-of-interest comparing circuit 
or routine 314 determines a composite value result for each region-of-interest pair, and then 
picks the RROIi and RROI 2 based on that composite value. Each composite value result 
incorporates the partial image results of each of the feature images Fk. In an exemplary 
embodiment, a criterion known as the Fisher distance or criterion is used to compare the 
partial filtered image results determined in each symmetrically located pair of regions-of- 
interest in each of the feature images Fk individually. The Fisher distance is a quotient with a 
numerator that is the squared difference between the means of two elements and with a 
denominator that is the sum of the variances of the two elements. Firstly, the Fisher distance 
is determined for two elements that are the feature image pixel data in the two regions-of- 
interest for each feature image Fk. Secondly, the composite value result for each region-of- 
interest pair is determined as the sum of the Fisher distances for that region-of-interest pair 
for all the feature images Fk. The region-of-interest pair having the largest composite value 
result is picked as the representative regions-of-interest RROIi and RROI2. It should be 
appreciated that an analogous Fisher distance procedure can be applied to the underlying 
feature image pixel data without determining individual Fisher distances for each feature 
image F k . 

[0051] Once the regions-of-interest comparing circuit or routine 314 selects the 
representative pair of regions-of-interest, RROIi and RROI 2 , the case-specific filter selection 
circuit or routine 350, previously discussed with reference to Fig. 1, selects the best case- 
specific filters from the candidate filters. Such filters are referred to as selected filters herein. 
The best case-specific filters are the filters that best emphasize or isolate the location of the 
edge in the current area of interest. 
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[0052] It should be appreciated that a particular candidate filter corresponds to a 
particular generated feature image Fk and to the associated partial filtered image results and 
overall filtered image result(s). It should be further appreciated that a selected filter j is 
effectively selected at the same time that a selected feature image Fj is selected. Thus, in 
various exemplary embodiments, the case-specific filter selection circuit or routine 350 
refines the candidate filter selections by selecting a subset of feature images Fj from the 
candidate set of feature images F^. The selection is based on consideration of the filtered 
image results corresponding to the RROIi and RROI2 of the candidate feature images Fk. 

[0053] The selection is done to reduce the number of filters that need to be applied 
to the original image, or similar images, in order to generate a pseudo-image which is useful 
for edge detection. Selecting only the most useful filters achieves faster edge detection and/or 
improves the accuracy and reliability of detecting the edge using the systems and methods 
according to this invention. In general, candidate filters are eliminated that do not 
significantly emphasize differences in the textures on the two opposite sides of the boundary 
in the area of interest. Specifically, candidate filters are eliminated that do not significantly 
emphasize differences in the textures in the RROIi and RROI 2 . 

[0054] In one exemplary embodiment, the regions-of-interest comparing circuit or 
routine 314 has determined the representative Fisher distance for the RROIi and RROI2 (the 
R-Fisher distance) of each candidate feature image Fk, as outlined above. In such a case, the 
case-specific filter selection circuit or routine 350 selects feature images Fj that have a 
significant R-Fisher distance, since a significant R-Fisher distance corresponds to a filter that 
is useful for emphasizing the boundary in the area of interest. In one exemplary embodiment, 
the R-Fisher distances for all candidate images Fk are compared and the maximum R-Fisher 
distance is determined. Then, all feature images/filters having an R-Fisher distance greater 
than 50% of that maximum R-Fisher distance are selected as the selected feature images Fj 
and/or selected filters j. In an extension of this embodiment, not more than the best five of 
the previously selected filters are retained as the selected filters. It is recognized that the 
selection technique just discussed does not produce an optimal subset of feature images Fj 
and/or selected filters j. In general, to obtain a "best" subset of feature images requires 
exhaustive methods that are processor-power and/or time consuming. Thus, exhaustive 
optimizing techniques are currently not desirable in applications for which the edge detection 
systems and methods according to this invention are intended. 
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[0055] It should be appreciated that any known or later developed filter selection 
technique could be used to select the subset of feature images Fj and/or selected filters j. It 
should also be appreciated that, while the subset of feature images Fj is less than the candidate 
set of feature images Fk, the subset of feature images Fj could be equal to the candidate set of 

5 feature images Fk. It should further be appreciated that once the case-specific filter selection 
circuit or routine 350 has selected the subset of feature images Fj and/or selected filters j, the 
region-of-interest comparing circuit or routine 314 can optimally be used to re-determine the 
RROIi and RROI2, this time based only on the selected feature images Fj. A different RROIi 
and RROI2 may result and should be the RROIi and RROI 2 used for subsequent case-specific 

1 0 operations. 

[0056] It should be also appreciated that there are a variety of alternative feature 
selection techniques usable by the systems and methods according to this invention, as will be 
apparent to one skilled in the art. Further, feature extraction techniques are well-known 
alternatives to feature selection techniques, and could be used instead of, or in addition to, 

IS, various operations of the case-specific filter selection circuit or routine 350 and the related 
operations of the elements 311-314 outlined above. See, for example, the chapter titled 
Feature Extraction and Linear Mapping for Signal Representation, in the book Introduction to 
Statistical Pattern Recognition, by Keinosuke Fukunaga, Academic, San Diego, 1990. 
r Furthermore, Sobel filters, Law's filters, Gabor filters as well as numerous alternative filters, 

20_ as well as their various uses and implementation to generate filtered images, feature images, 
feature vectors, classification vectors, feature extraction, and pseudo-images and the like, are 
also known to one skilled in the art. See, for example, "Filtering for Texture Classification: A 
Comparative Study", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 
21, No. 4, April 1999; See generally feature selection and extraction, Statistical Pattern 

25 Recognition, by Andrew Webb, co-published in the USA by Oxford University Press Inc., 
New York, 1999; "Rapid Texture Identification," Proc. SPIE Conf. Image Processing for 
Missile Guidance, pp. 376-380, 1980; and "Unsupervised Texture Segmentation Using Gabor 
Filters," Pattern Recognition, vol. 24, no. 12, pp. 1,167-1,168, 1991. 

[0057] Furthermore, although various exemplary embodiments of the systems and 

30 methods according to this invention are described herein as determining or extracting images, 
filtered images, feature images, and/or pseudo-images, as well as determining various partial 
filtered image results, filtered image results, and image metrics usable to evaluate and 
compare these various image types, it should be appreciated that these terms are not mutually 
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exclusive in various embodiments according of the systems and methods according to this 
invention. For example, as is apparent from the nature of the mathematic transforms and 
algorithms employed herein, a portion of a filtered image or feature image may also operate 
as, or be derivable from, a related partial filtered image result. Thus, these terms are used in 
various contexts herein for the purpose of describing of various operations, but are not 
intentionally used in a mutually exclusive sense. 

[0058] In particular, various operations herein are described as determining one or 
more feature images, partial filtered image results, and/or filtered image results. Various 
other operations are described as making a selection based on or between the previously 
determined images and/or results. It should be appreciated that the dividing line between 
related determining and selecting types of operations is largely arbitrary. For example, it is 
clear that a more primitive feature image, partial filtered image result, and/or filtered image 
result could be selected by a more refined selector which compensates for any deficiencies of 
the more primitive element in order to achieve the objectives of this invention. Conversely, it 
is clear that a more primitive selector may be used with more refined feature images, partial 
filtered image results, and/or filtered image results which compensate for any deficiencies of 
the more primitive selector in order to achieve the objectives of this invention. Thus, it 
should be appreciated that in various exemplary embodiments, the various operations 
associated with "determining" and "selection" may be interchanged, merged, or 
indistinguishable. 

[0059] Once the case-specific filter selection circuit or routine 350 has selected the 
subset of feature images Fj and/or selected filters j, the pseudo-image generating circuit or 
routine 360, previously discussed with reference to Fig. 1, operates to generate a pseudo- 
image based on the selected filters j, also called case-specific filters herein. 

[0060] In one exemplary embodiment, if a set of normalized feature images Fj are 
not currently generated or available from the memory 130, the pseudo-image generating 
circuit or routine 360 causes the feature image generating circuit or routine 3 12 to generate a 
set of normalized feature images Fj based on the subset of case-specific filters j, according to 
previously described operations. The pseudo-image generating circuit or routine 360 then 
determines a pair of classification vectors CV1 and CV2 corresponding to the RROIi and the 
RROI2, respectively. 

[0061] The classification vector CV1 can include the mean value of the pixel data in 
the RROIi of each of the normalized feature images Fj corresponding the case-specific filters 
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j. Thus, the dimension of CV1 is n, where n is the number of case-specific filters j selected 
by the case-specific filter selection circuit or routine 350 as outlined above. CV2 is a similar 
vector, similarly determined, based on the pixel data in the RROI2 of each of the normalized 
feature images Fj. After the classification vectors CV1 and CV2 have been determined, the 
pseudo-image generating circuit or routine 360 generates the pseudo-image that will be used 
in performing the current set of edge location operations. The pseudo-image is generated for 
at least the previously described area of interest. This exemplary embodiment is based on 
comparing the data of the normalized feature images Fj to the classification vectors CV1 and 
CV2. 

[0062] A classifier can be used by the pseudo-image generating circuit or routine 
360 to generate the pseudo-image. The classifier can be a data clustering technique where, in 
this case, a feature vector, i.e., also called a pixel feature vector , corresponding to the spatial 
location of a pixel in the area of interest is determined to belong to a cluster or region 
specified by a membership grade. As used herein, a pixel feature vector (PFV) includes the 
feature pixel values for a corresponding spatial location in each of the normalized feature 
images Fj corresponding the case-specific filters j. Thus, the dimension of a pixel feature 
vector is n, where n is the number of case-specific filters j selected by the case-specific filter 
selection circuit or routine 350 as outlined above. Furthermore, the elements of the PFV's are 
ordered similarly to the elements of CV1 and CV2, and are based on the same underlying 
feature image pixel data, for example, normalized feature image pixel data. Thus, 
corresponding elements of the PFV's and CV1 and CV2 may be meaningfully compared. 

[0063] Each pixel location of at least the area of interest is in turn selected by the 
pseudo-image generating circuit or routine 360. The classifier is applied by the pseudo-image 
generating circuit or routine 360 to the corresponding pixel feature vector to determine 
whether that pixel feature vector is more like the CV1 corresponding to ROIi, or more like 
the CV2 corresponding to ROI2. For example, the Euclidean distance may be used to 
determine respective "distances" between a current PVF and CV1 and CV2, respectively. 
The Euclidian distance to CV1 or CV2 is the sum of the squares of the differences between 
corresponding elements of the current PVF and CV1 or CV2, respectively. The smaller the 
Euclidian distance, the more the two vectors compared by that Euclidian distance resemble 
each other. Based on the Euclidian distance, or the component elements of the Euclidian 
distance, a membership value is determined and assigned to the pixel of the pseudo-image 
that corresponds the currently evaluated pixel feature vector. 
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[0064] In a sense, the pseudo-image pixel value indicates the degree to which that 
pixel "belongs" on the side of the border of RROIi or on the side of the border of RROI 2 - In 
an exemplary embodiment, each pseudo-pixel is assigned a value between 0.0, which 
represents complete membership to the side of the border of RROIi, and 1.0, which represents 
complete membership to the side of the border of RROI 2 . 

[0065] In one particular embodiment, the membership values are determined using a 
fuzzy c-means classifier modified as described below, based on a fuzzy c-means classifier 
described in the article "FCM: The fuzzy c-Means Clustering Algorithm", Computers & 
Geosciences, Vol. 10, No. 2-3, pp 191-203, 1984, which is incorporate herein by reference. 
Using the symbols as defined in that article, the classifier parameters are set as c=2 (two 
clusters), m=2 (weighting exponent), v=CVl, CV2, as defined herein (vectors of centers), 
norm=Euclidean distance, n=number of data= number of pixels in the tool area of interest. In 
a preferred modified version of this algorithm, there are no iterations and the clustering is 
done with initial centers that are the clusters v= CV1 ,CV2. Because well-defined prototype 
clusters CV1 and CV2 are used, clustering may be stopped after the first iteration, i.e. the first 
classification, and good results are still obtained. It should be appreciated that this set of 
parameters produces a non-linear classification, emphasizing membership value variations 
near the boundary. 

[0066] In general, this fuzzy clustering algorithm produces two membership images: 
The first one is the membership value of each pixel to cluster 1 and the second is the 
membership value of each pixel to cluster 2. However, because the sum of memberships for 
each pixel location must be unity for our case, the membership images are complementary 
and we need only determine one of them. 

[0067] It should be appreciated that there are a wide variety of alternatives for generating 
various pseudo-images based on a set of feature images. Such alternatives include alternative 
fuzzy classifiers, neural classifiers, a hidden mark-up model or any other now known or later- 
developed technique or algorithm which is capable of generating a set of pseudo-image pixel 
values usable in accordance with this invention. Furthermore, when another type of 
classification or pseudo-image generation is performed, it should be appreciated that the 
membership function operations described above may be replaced by any other appropriate 
operations for applying weighting factors to the various filtered image results or feature image 
results corresponding to each pixel location, in order to accord them greater or lesser values 
based on their similarity to the characteristics of RROII or RROI2. Various alternatives 
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usable with the system and methods of the invention will be apparent to one skilled in the art. 

[0068] Once the pseudo-image generating circuit or routine 360 has generated a 
current pseudo-image, the edge point analyzing circuit or routine 370, as previously discussed 
5 with reference to Fig. 1 , can operate to determine one or more edge points along the boundary 
in the area of interest. In various exemplary embodiments of the edge point analyzing circuit 
or routine 370, elements 377-379 can operate as follows: 

[0069] The scan line determining circuit or routine 377 can determine one or more 
edge-detection scan lines and the direction or polarity of "traversing" the scan lines in a 
10 known manner such as that used in commercially-available machine vision systems, such as 
the QUICK VISION™ series of vision inspection machines and QVPAK™ software 
ri available from Mitutoyo America Corporation (MAC), located in Aurora, IL. Generally, the 
- J scan line determining circuit or routine 377 determines the scan lines based on the data 
CO associated with an appropriately located edge tool on the input image and/or an operation of 
1 5yp the area of interest generator 150. The operator input may influence the spacing of the scan 
=~ lines, or a default value such as 5 or 20 pixel units, or a percentage of a width of the area of 
s interest can be automatically set as a default value. The scan lines extend across the boundary 
%4, in the pseudo-image. The direction or polarity of traversing the scan lines to perform edge 

detection operations is determined based on the pseudo-image characteristics in the vicinity of 
202 the edge. The direction of traversing the scan lines can generally proceed from a region with 
less variation to a region with more variation. More generally, the direction of traversing the 
scan lines should proceed in that direction that provides edge detection results that are less 
noisy. 

[0070] The edge point detection circuit or routine 378 estimates an edge point along 
25 each scan line determined by the scan line determining circuit or routine 377, according to 
any now known or later-developed set of edge-point detection operations. The values along 
each scan line in the pseudo-image constitute a one-dimensional signal. In one exemplary 
embodiment the edge point is a point of maximum gradient along the scan line signal in the 
pseudo-image. It should be appreciated that any known or later developed edge detection 
30 operation used on gray-scale intensity images and the like may be applied to detect and 
estimate the edge position in the pseudo-image. 

[0071] The edge point detection circuit or routine 378 may also record one or more 
edge detection parameters associated with the estimated edge points in one or more portions 
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of the memory 130, so that a case-specific edge-detection operation can be run automatically 
using the recorded parameters for edge-detection and/or edge point reliability evaluation. 
Such parameters may include various characteristics of the scan line pseudo-pixel value 
profiles which characterize the edge, such as the pixel value change across the edge, the 
direction of pixel value increase across the edge, the number or proportion of scan lines 
across the edge that include pixel value changes above a threshold value, and the like. In one 
exemplary embodiment, the mean value of each characteristics is the value recorded as the 
basis for case-specific automatic "run-time" edge measurements performed later. This will 
tend detect only those edge points that have a fairly high initial reliability. 

[0072] The edge point refining circuit or routine 379 may then perform operations to 
refine one or more initial edge point estimates, based on additional information. In one 
exemplary embodiment the edge point refining circuit or routine 379 performs an analysis 
operation on a plurality of pixel locations in a local region extending on both sides of an 
initially estimated edge point along a direction generally parallel to the scan line. 
In one exemplary operation, data associated with a number of closest pixel locations q along a 
selected detected edge point's scan line are used to refine the position of the initially estimated 
edge point. For each pixel location i of the q pixel locations surrounding the initially 
estimated edge point, the edge point refining circuit 379 calculates the Euclidian distance, 
discussed above, between the (i+1) pixel location and the (i-1) pixel location based on those 
particular pixel locations in a current set of feature images produced by the feature image 
generating circuit or routine 312 and selected by the case-specific filter selection circuit 350. 
These Euclidian distance values located at each of the q pixel locations form a curve. The 
analysis operation then determines a centroid location for the area under the curve. The 
centroid location is in terms of the pixel locations, and thus determines the refined edge point 
estimate along the scan line. In one exemplary embodiment, the edge point refining circuit or 
routine 379 refines each initial edge point estimate using the centroid location operations. 

[0073] The edge point refining circuit or routine 379 may also perform operations 
such the operations described below with reference to the steps SI 651 -SI 662 of Fig. 14, 
and/or the steps S2520-S2560 of Fig. 16. for the purpose of validating initially determined 
edge points and increasing their reliability. In various other exemplary embodiments, the 
edge point refining circuit or routine 379 interacts with the boundary locating and refilling 
circuit or routine 380, which determines the edge points to be refined, as previously described 
with reference to Fig. 1. 
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[0074] The edge point refining circuit or routine 379 may also revise one or more 
edge detection parameters previously determined and/or recorded by the edge point detection 
circuit or routine 378. It may also add or record one or more edge additional edge detection 
parameters associated with the refined edge points in one or more portions of the memory 
130, so that a case-specific edge-detection operation can be run automatically using the 
recorded parameters for edge-detection and/or edge point reliability evaluation. 

[0075] In various exemplary embodiments of the boundary locating and refining 
circuit or routine 380, elements 381-383 can operate as follows: 

[0076] The shape analysis circuit or routine 381 analyzes a plurality of estimated 
edge points to determine if they correspond to criteria for a reliable edge detection. In one 
exemplary embodiment, the criteria includes a threshold value for a shape score based on the 
deviation between a line (which may be a curved line) fit to the estimated points and an 
expected edge shape; a threshold value for a location score based on the deviation between 
the line fit to the estimated points and an expected edge location; and an outlier threshold 
value based on the standard deviation of the individual edge point distances from the line fit 
to the estimated edge points. The expected edge shape and location are set by the operator of 
the vision system using edge tool selection and placement, or by other user input, or 
automatically based on various CAD data operations. Based on the results of the operations 
of the shape analysis circuit or routine 381, the outlier elimination circuit 382 selects one or 
more edge points failing the outlier threshold value criterion for elimination or refinement. In 
various exemplary embodiments, the edge point refining circuit or routine 379 performs the 
edge point estimate refinement as previously described, and the shape analysis circuit or 
routine 381 and the outlier elimination circuit 382 recursively analyze the plurality of 
estimated/refined edge points until the remaining estimated edge points are finally determined 
to constitute a reliable or unreliable edge. For an unreliable edge, the outlier elimination 
circuit outputs a corresponding error signal on the data/control bus 140. It should be 
appreciated that in various exemplary embodiments, the operations of the shape analysis 
circuit or routine 381 and the outlier elimination circuit 382 may be merged or 
indistinguishable. For a reliable edge, the edge location determining circuit 383 determines 
the final edge location data, which may include the final estimated edge points and/or other 
derived edge location parameters and outputs the data on the data/control bus 140 to one or 
more portions of the memory 130 and/or the input/output interface 110. 
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[0077] In various exemplary embodiments of the edge mode determining circuit or 
routine 390, elements 391-392 can operate as follows: 

[0078] The edge tool interpreting circuit or routine 391, which, for each particular 
edge case, determines the appropriate mode of operation for the various other elements 
performing the edge detection operations based on the edge tool data associated with that 
particular edge case. The appropriate mode of operation is based on whether the particular 
edge in the area of interest is appropriately analyzed by edge detection operations applied to 
the input image or edge detection operations applied to a pseudo-image, as previously 
described. In a first exemplary embodiment, unique edge tools are exclusively associated 
with input image edge detection for well-defined edges and pseudo-image edge detection for 
significantly textured edges, respectively. In such a case, the edge tool interpreting circuit or 
routine 391 interprets the type of edge tool associated with a current edge case and operates 
accordingly. In a second exemplary embodiment, the edge tools include secondary selectable 
features, such as a check box or the like, which are exclusively associated with input image 
edge detection for well-defined edges and pseudo-image edge detection for significantly 
textured edges, respectively. In such a case, the edge tool interpreting circuit or routine 391 
interprets the secondary edge tool feature associated with a current edge case and operates 
accordingly. 

[0079] However, in various other exemplary embodiments, one or more edge tools 
can have no characteristic or feature that is exclusively associated with input image edge 
detection for well-defined edges or pseudo-image edge detection for significantly textured 
edges. In such cases, the area of interest analyzing circuit or routine 392 can determine the 
appropriate edge detection mode. Here, the area of interest analyzing circuit or routine 392 
can automatically determine at least one texture characteristic, such as a local variability 
value, in evaluation regions on both sides of the edge in the area of interest. The location of 
the evaluation regions is based on the data associated with an appropriately located edge tool 
and/or an operation of the area of interest generator 1 50. The area of interest analyzing circuit 
or routine 392 then can automatically select the appropriate mode of edge detection based on 
the determined texture characteristics and establishes the appropriate mode of operation for 
the various other elements performing the edge detection operations for that particular edge 
case. 

[0080] Fig. 3 illustrates two images of exemplary objects having a significantly- 
textured edges that can be detected and located using the edge detection systems and methods 
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according to this invention. The image 400 includes an edge/boundary 406 that can be 
precisely located with various embodiments of the boundary detection or edge detection 
systems and methods according to this invention. The image 400 has an edge/boundary 406 
that exists between a first portion 402 of the image 400 and a second portion 404 of the image 
5 400. The image 400 is the image of an object that has been captured by the vision system 1 0 
as described with reference to Fig. 1 . 

[0081] Before the edge detection systems and methods of this invention can be used 
in an automated mode to locate edges or boundaries during a run mode, the edge detection 
systems and methods according to this invention must be set up to detect specific edges using 
10 specific image-derived parameters. Using an image that has been captured by the vision 

system 10, that acquired image is used by the edge location operation as an input image 500. 
q Fig. 3 shows one exemplary embodiment of the input image 500 that can be used with the 
" :] edge detection systems and methods of this invention. The input image 500 has an edge 506 
that is defined in between a first portion 502 and a second portion 504 of the input image 500. 
15=1} [0082] After the input image 500 is acquired, the input image 500 is displayed on 

~Z the display 102 so that a user can define an area-of-interest using a graphical user interface 

and positioning a boundary detection tool, also referred as an boundary tool or edge detection 
M= tool, on a particular edge or portion of an edge to be detected. The area-of-interest is defined 
by the area of interest generator 1 50 based on the data corresponding to the positioned edge 
20 tool. One exemplary boundary tool 508 includes a box 505 configurable by the user to 

outline and determine the area-of-interest. For example, the box may be configured in an arc 
or circle shape, or in the shape of a rectangle as shown in Fig. 3. However, it should be 
appreciated that the boundary detection tool 508 can be drawn in any shape that allows an 
area of interest to be defined by the user or an automated process. The boundary tool 508 also 
25 includes region-of-interest indicators 512 shown as overlapping identical rectangles in Fig. 3. 
In various other embodiments, the edge tool is an edge point tool, and the area of interest and 
the region of interest indicators are not indicated on the display, but are determined 
automatically by the previously described area of interest generator 150 and the filtered image 
analyzing circuit 310, respectively, based on a simple point cursor positioned by the user. 
30 Various other exemplary edge tools are apparent in the previously referenced commercially- 
available machine vision system and the like. 

[0083] After the boundary detection tool 508 has been drawn on the input image 
500, the user can define a point-of-interest (P0) within the area-of-interest bounded by the 
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boundary tool 508. Alternatively, the point of interest P0 is automatically determined relative 
the position of the boundary detection tool 508 and may not be visible on the display. The 
point of interest P0 is generally, or possibly, only indicative of a point on the boundary, or 
edge. The user can also direct the edge location operation to focus on the point P0. 
5 Moreover, the user can define a distance between various "scan" lines 509 extending across 
the boundary in the area of interest. Alternatively, based on the previously discussed 
boundary detection tool operations and information, the previously described edge point 
analyzing circuit 370 can automatically determine the distance between the scan lines 509 and 
the end points, i.e., (xl, yl), (x2, y2) of each scan line 509 extending across the boundary in 

1 0 the area of interest. Similarly the previously described filtered image analyzing circuit or 

routine 310, can automatically determine the locations of the regions-of-interest indicated by 
the region-of-interest indicators 512. Thus, operations associated with the boundary detection 
* tool 508 can be manually defined by user input or by an automated process using predefined 
ffl boundary detection tool characteristics. By allowing the user to select a boundary detection 

1 5 tool having predefined characteristics, boundary detection operations can be directed by 
= ; operators having little or no understanding of the underlying mathematical or image 
processing operations. 

[0084] In Fig. 4, the boundary detection tool 508, scan lines 509 and regions-of- 
interest indicators 5 12 are illustrated relative to yet another input image 600. For purposes of 

20*:? clarification, Fig. 4 illustrates another exemplary set regions-of-interest, indicated by the 
regions-of-interest indicators 512, that are generated by and usable by the systems and 
methods according to this invention. It should be appreciated that the regions of interest 
indicators are not displayed in some embodiments, and that a region of interest originally 
generated relative to an input image also comprises a spatially congruent region of interest in 

25 any other corresponding filtered image, feature image, pseudo-image, or the like, described 
herein. As previously described, the regions-of-interest can be determined automatically or 
the user can determine them by dragging and dropping displayed region of interest indicators 
512, for example. As previously described, the regions of interest may be arranged in 
symmetric or approximately symmetric regions-of-interest pairs 514 around the central point 

30 P0. Figure 4 shows 4 regions-of-interest pairs. Furthermore, in an alternative to the 

previously described automatic operations for determining the representative regions of 
interest RROl! and RROI 2 , the user can select an RROIi and an RROI 2 that are located on 
opposite sides of the point-of-interest P0 and arranged along a line generally perpendicular to 
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the boundary and within of the area-of-interest. However, it should be appreciated that the 
best RROIi and an RROI2 will not generally or necessarily be a regions-of-interest pair 
arranged along a line generally perpendicular to the boundary. 

[0085] Fig. 5 illustrates one exemplary embodiment of a pseudo-image 700 
5 generated by the pseudo-image generating circuit or routine 360, as previously described. It 
should be appreciated that the pseudo-image need not be displayed, and is generally not 
displayed by the systems and methods according to this invention. 

[0086] More generally, various exemplary embodiments of the systems and methods 
according to this invention are described herein as generating a various "images" as the basis 
10 for an image result which is evaluated. However, it should be appreciated that the image 

result may be determined from a variety of data representations not generally recognized as an 
"image". Provided that such data representations are usable to provide one or more image 
' : results which are usable according to the systems and methods of this invention, such data 
f J representations are included in the scope of the terms "feature image" or "pseudo-image", and 
15- J the like, and thus are within the scope of the systems and methods according to this invention. 
5 It should be further appreciated that, in various other exemplary embodiments, depending on 
y - the image results to be determined, the image result may be determined directly from the 
M» input image and the appropriate candidate or selected filters without needing to represent or 
y ;: generate recognizable image as a recognizable intermediate step. 
2.0 . [0087] Nevertheless, the pseudo-image 700 is useful for purposes of clarification. 

|M= As previously described, the pseudo-image 700 is spatially congruent with the input image, 
and thus with the various tool elements and regions of interest previously described with 
reference to Figs. 3 and 4. It should be appreciated that the particular pseudo-image 700 
corresponds to a magnified input image and therefore can support high-accuracy edge 
25 location despite the blurred appearance of this particular image. The direction of traversing 
the scan lines 509, as indicate by the arrowheads on the scan lines 509, can be determined as 
previously described. The pseudo-image 700 need only be determined in the area of interest, 
bounded by the line 704 in Fig. 5. The edge points 702, indicated "x's" along the 
edge/boundary 706 in the pseudo-image 700 are determined as previously described. Because 
30 the pseudo-image is spatially congruent with the input image, the edge points determined for 
the pseudo-image are easily displayed on a graphical user interface including the input image, 
in various exemplary embodiments of the systems and methods according to this invention. 
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[0088] Fig. 6 illustrates one exemplary embodiment of multiple edge locations 802, 
determined for an exemplary input image 800, detected by an exemplary edge point analyzing 
circuit or routine 370 employing the previously described gradient-type edge detection 
operations. Because the pseudo-image is spatially congruent with the input image, the edge 
points 802 determined for the pseudo-image are easily displayed as the edge points 802 on a 
graphical user interface including the input image, in various exemplary embodiments of the 
systems and methods according to this invention. The regions-of-interest indicators 814 and 
the limits of a boundary tool 808 are also shown in Fig. 6. 

[0089] In a part-programming or training mode of the vision system 10, in an 
exemplary embodiment, a display including elements such as the elements 800, 802, 808, for 
example is displayed to the user once the edge points 802 have been determined. If the user 
approves of the displayed edge points 802 and any associated edge location data that may also 
be generated and output, the user accepts the results through one or more actions, which may 
be as little as moving on to performing a new operation with the vision system 10. Once user 
acceptance is indicated by any means, the control system portion 100 stores the various 
previously describe operations and parameters used to determine the edge points 802 as a 
case-specific routine or a case-specific trained edge/boundary detection tool in the part 
program memory portion 133. The control system portion 100 may also store the associated 
edge location data that was generated and output, in the memory 130. The case-specific 
routine or trained edge/boundary detection tool stored by the control system portion 100 is 
generally stored and/or included in one more part programs, and are usable to automatically, 
quickly and reliably detect and locate edges in similar cases in a "run mode". The similar 
cases where the case-specific routine and/or trained edge/boundary tool may be 
advantageously usable include, for example, cases such as locating the identical edge in the 
future, locating another portion of the same edge on the same part, i.e., in a different field of 
view, locating the "same" edge on a future part produced according to the same 
specifications, and locating other edges made by the same process, such as edges on a variety 
of similar holes in various locations on a flat sheet, such as printed circuit board holes. These 
and other type of similar-edge cases will be apparent to those skilled in the art and to typical 
user of machine vision systems and according these examples are in no way limiting. 

[0090] A more detailed description of the run mode process will be described with 
reference to Figs. 15 and 16 
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[0091] Fig. 7 is a flowchart outlining one exemplary embodiment of a method for 
training a boundary detection tool to detect a specific case of an edge in an input image 
according to this invention. A trained boundary detection tool can be usable by a fast and 
reliable automatic boundary detection routine, such as may be included in a part program for 
inspecting similar cases of edges on similar parts. After beginning operation in step SI 000, 
operation proceeds to step SI 100, where a first or next input image is acquired. Then, in step 
S1200, an area of interest within the input image is determined and scan lines extending 
across the determined area of interest are determined. Next, in step SI 300, one or more 
feature images of at least the area of interest are generated. Operation then continues to step 
S1400. 

[0092] In step SHOO, those feature images generated in step SI 300 are analyzed to 
determine and select those feature images that are usable to distinguish a first region-of- 
interest, on one side of the specific edge to be detected from a second region-of-interest on 
the other side of the specific edge to be detected. As outlined above, some of the generated 
feature images, in view of a selected representative pair of regions-of-interest, may not have 
sufficiently different feature pixel values on the two sides of the edge to support reliable edge 
detection. In step SHOO, the initial set of feature images can be reduced if any of the feature 
images would not be useful in improving the edge detection. 

[0093] Next, in step SI 500, a membership image is generated that indicates the 
membership value of each pixel in at least the area of interest in relation to two clusters. The 
centers of the two clusters are based on the characteristics of the selected representative pair 
of regions-of-interest selected in step SHOO. The membership values are based on the cluster 
center characteristics and the feature images generated in step SI 300 and selected in step 
SHOO. The two clusters used in creating the membership image represent the two types of 
feature image data on each side of the edge to be detected reflected in the selected feature 
images selected in step SHOO. Then, in step SI 600, edge points along the scan lines are 
determined based on the membership image generated in step SI 500, and "good" edge points 
are selected from the detected edge points. Operation then proceeds to step SI 700. 

[0094] In step S 1 700, for each kept detected edge point from step S 1 600, a close 
"neighborhood" of the detected edge point is analyzed to correct the location of the detected 
edge point and a group of detected edge points is analyzed to eliminate outliers. In step 
SI 700, operations such as one or more of the operations previously described with reference 
to the edge point refining circuit 379 and the boundary locating and refining circuit 380 are 
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performed. In one exemplary operation, data associated with a number of closest pixel 
locations q along a selected detected edge point's scan line are used to refine the position of 
the selected detected edge point. For each pixel location i of the q pixel locations 
surrounding the selected detected edge point, the edge point refining circuit 379 calculates the 
Euclidian distance between the (i+1) pixel location and the (i-1) pixel location based on those 
particular pixel locations in the current set of feature images. These Euclidian distances for 
each of the q pixel locations form a curve. Subsequently, the centroid of the curve is used as 
the refined location of that selected detected edge point. The boundary locating and refining 
circuit 380 analyzes a group of selected detected edge points to detect and correct or eliminate 
outliers. Next, in step SI 800, the boundary detection tool data that represents the information 
determined to detect this specific case of edge in input image that has been created in the 
training mode is accepted and/or stored. Acceptance may be determined by the user based on 
a display of the final set of edge points or associated boundary location data. As a default 
condition, the boundary detection tool data may be stored without specific acceptance. Next, 
in step SI 900, a determination is made whether another input image is to be acquired If 
another image is to be selected and analyzed, then operation returns to step SI 100. 
Otherwise, operation proceeds to step SI 950 where operation of the method stops. 

[0095] Fig. 8 is a flowchart outlining in greater detail one exemplary embodiment of 
the method for determining an area-of-interest of step SI 200. After the operation begins in 
step SI 200, operation proceeds to step S1210 where the user determines whether the edge 
location operation will use an automatic boundary detection tool to reflect an area of interest 
within or through which the specific edge to be detected. If the user will not use an automatic 
boundary detection tool, operation proceeds to step SI 220. Otherwise, operation jumps to 
step SI 250. In step SI 220, the user manually draws and/or edits a boundary detection tool as 
previously described above to select the boundary to be located and the desired area of 
interest. Then, in step SI 230, the user selects a point P0 within the area of interest bounded 
by the created boundary detection tool, and preferably close to the boundary, to focus the edge 
detection process. It should be appreciated that the point P0 may also be generated as part of 
the process of drawing a tool, and the operations of steps SI 220 and SI 230 may be 
indistinguishable. In step S1240, the scan lines' position or spacing along the boundary, and 
the lengths or the end points of the scan lines, are determined by user input or by default 
positions derived from the selected area of interest. Operation then jumps to step SI 260. 
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|0096] In contrast to steps S 1220, S 1230, and 1240, in step S 1250, an automatic 
boundary detection tool is used. Various automatic boundary detection tools may have 
various scopes of operations. As one example, the user may select an appropriate tool, such 
as a point tool, or a box tool, and then do as little as "position" a cursor/pointer element of the 
tool near a point intended as "P0" and the tool will then automatically determine any of the 
previously discussed tool parameters which are required for edge detection using that tool. 
Scan lines can also be automatically defined. Operation then continues to step SI 260. Then, 
in step SI 260, operation returns to step SI 300. 

[0097] Fig. 9 is a flowchart outlining in greater detail one exemplary embodiment of 
the method for generating feature images of step S1300. Beginning in step S1300, operation 
proceeds to step S13 10, where a determination is made whether the user will select a 
candidate filter group manually or have the candidate filter group automatically determined. 
As previously discussed, the term candidate filter implies that the filter will be used in 
generating a filtered image result from a current image, but that it will be accepted or rejected 
later, based on the image result. If the candidate filter group will not be set automatically, 
then operation proceeds to step S1320. Otherwise, operation jumps to step S1330. The 
determination to automatically select the candidate filter group can be made and/or 
communicated using a candidate filter method option of a graphical user interface. 

[0098] In step SI 320, the user manually selects a candidate filter group as 
previously discussed above. Operation then jumps to step SI 340. In contrast, in step 1330, 
the candidate filter group to be used is automatically determined. Then, operation proceeds to 
step SI 340. 

In step SI 340, the candidate filters selected or automatically determined through the 
candidate filter method are applied to the defined area-of-interest of the input image to 
generate a corresponding number of feature images. Then, in step SI 350, operation returns to 
step SHOO. 

[0099] Fig. 10 is a flowchart outlining in greater detail one exemplary embodiment 
of the method for performing the useful feature image selection of step SHOO. As previously 
discussed, when a useful feature image is selected a corresponding filter used in generating 
the feature image is also effectively selected. Beginning in step SHOO, operation proceeds to 
step SI 410, where a single pair of regions-of-interest, or one or more pairs of regions-of- 
interest, such as the various pairs of regions-of-interest shown in Figs. 3, 4 and 6, are defined. 
In particular, for each pair of regions-of-interest, a first region of interest is defined on one 
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side of the point of interest PO within the area of interest bounded by the boundary detection 
tool. The second region of interest of that pair of regions-of-interest is defined diametrically 
on the other side of the point of interest PO from the first region of interest of that pair of 
regions-of-interest. Then, in step SI 420, a representative pair of regions-of-interest RROIi 
5 and RROI2 is selected from the one of the one or more pairs of regions-of-interest. Of course, 
it should be appreciated that step SI 420 can be omitted if only a single pair of regions-of- 
interest is defined in step S1410. 

[0100] Next, in step S 1430, a subset of the feature images, which generally includes 
the feature images that best distinguish between the image data within the representative pair 

10 of regions-of-interest that are on opposite sides of the selected point P0, is selected based on 
an analysis of the feature image data within the representative pair of regions-of-interest. The 
corresponding set of selected filters is at least temporarily stored as tool-related data. As 
outlined above, in various exemplary embodiments, this selection is done to reduce the 
number of filters that need to be applied for edge detection, in order achieve faster edge 

1 5 detection and/or to improve the accuracy and reliability of detecting the edge using the 

systems and methods according to this invention.. Operation then continues to step S1440. 

[0101] The step S1430 constitutes a feature selection step. It should be appreciated 
that feature extraction is a well-known alternative or supplement to feature selection. Feature 
extraction is a technique that, in effect, combines the feature images to generate a smaller but 

20 more effective set of feature images. Various usable feature extraction methods will be 

apparent to one skilled in the art and in various exemplary embodiments, feature extraction is 
performed in the step SI 430, instead of feature selection. Usable feature extraction methods 
are explained in the previously cited references. 

[0102] In step S 1 440, the representative pair of regions-of-interest, is re-selected to 

25 provide a later RROIi and RROI 2 based on the selected subset of feature images. It should be 
appreciated that step SI 440 is optional, and thus can be omitted. Next, in step SI 450, a 
number of classification vectors, such as, for example, the classification vectors CV1 and 
CV2 discussed above are created based on the image data in the latest representative pair of 
regions-of-interest RROIi and RRObof each of the feature images of the subset of feature 

30 images. In one exemplary embodiments, the mean image data in each of the feature images 
of the subset of feature images that lies within the representative regions-of-interest RROIi 
and RROI2 are calculated to generate the classification vectors CV1 and CV2, respectively. 
In general, the dimension of the classification vectors CV1 and CV2 is n, wherein n is the 
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number of feature images in the subset of feature images. Optionally, the latest RROI1 and 
RROI2 are in various exemplary embodiments, stored at least temporarily as tool-related data. 
Then, in step SI 460, operation returns to step SI 500. 

[0103] Fig. 1 1 is a flowchart outlining in greater detail one exemplary embodiment 
of the method for determining the membership-image of step SI 500. Beginning in step 
S 1 500, operation proceeds to step S 1 5 1 0, where a first or next pixel, that is, a pixel location, 
within at least the area of interest bounded by the boundary detection tool is selected. Next, 
in step SI 520, a membership value for the current pixel is determined using a classifier such 
as the previously described modified fuzzy c-means classifier and the created classification 
vectors CV1 and CV2. Then, operation proceeds to step SI 530. 

[0104] It should be appreciated that the modified fuzzy c-means classifier is just one 
exemplary classifier usable in the operations performed in the step SI 520 that is particularly 
fast and suitable when the operations of the steps S1420-S1450 shown in Fig. 10 have been 
performed. In various exemplary embodiments of the systems and method according to this 
invention, an "un-modified" fuzzy c-means classifier described in the previously cited 
reference is used. Such a classifier does not require prototypes of the clusters and works 
iteratively to improve the classification of the data points. Thus, there is no need to perform 
the operations of at least the steps S1420-S1450 shown in Fig. 10. 

[0105] Next, in step S1530, a determination is made whether any remaining 
unselected pixels need to be analyzed. If so, then operation proceeds back to step S1510. 
Otherwise, operation proceeds to step SI 540, where the direction of traversing along the scan 
lines to perform edge detection is determined. As previously discussed, the direction of 
movement along the scan lines can be determined using the membership image and the 
representative pair of regions-of-interest RROIi and RROI 2 used in determining the 
membership image. Then, in step SI 550, operation returns to step SI 600. 

101 06] It should be appreciated that the operations of the step S 1 540 can 
alternatively be omitted in the step SI 500 and performed at the beginning of step SI 600 
instead. In yet other exemplary embodiments, the operations of the step 1540 are omitted 
entirely and a default traversing direction is used. Although the reliability and accuracy may 
be somewhat affected for some edges, significant benefits will be retained in such 
embodiments of the systems and methods according to this invention. 

[0107] Fig. 12 is a flowchart outlining in greater detail one exemplary embodiment 
of the method for detecting and selecting edge point locations of step SI 600. Beginning in 



34 

step SI 600, operation proceeds to step S1610, where a first or next scan line is selected. 
Then, in step SI 620, one (or more) edge points within the selected scan line is detected using 
the membership-image defined in step SI 500. It should be appreciated that the pixel values 
of the original membership-image could be scaled, or normalized to an expected range if this 
is more advantageous or robust for the edge detection operation selected for use in the 
systems and methods according to this invention. Next, in step SI 630, the detected edge 
points are added to an initial set of edge points PEL Operation then continues to step SI 640. 

[0108] In step SI 640, a determination is made whether there are any remaining 
unselected scan lines. If so, operation proceeds back to step SI 610. Otherwise, operation 
proceeds to step S1650. In step S1650, valid edge points are selected based on the 
membership-image. Then, in step SI 670, operation returns to step SI 700. 

[0109] Fig. 13 is a flowchart outlining in greater detail one exemplary embodiment 
of the method for selecting a representative pair of regions-of-interest of the step SI 420. 
Beginning in step SI 420, operation continues to step SI 421, where, for a first/next pair of 
regions-of-interest, a similarity distance between the feature images data in the two regions- 
of-interest is determined based on each feature image of the candidate set of feature images. 
The similarity distance is, in various exemplary embodiments, the Fisher distance, which is 
discussed above. It should also be appreciated that several similarity distances could be 
determined. Then, in step SI 422, a determination is made whether the similarity distance has 
been determined for all of the pairs of regions-of-interest that have been defined. If so, 
operation continues to step SI 423. Otherwise, operation jumps to step SI 421 where 
similarity distance results are determined for the next pair of regions-of-interest. 

[0110] In step S1423, a representative pair of regions-of-interest, RROIi and RROI 2 , 
is selected based on the determined similarity distances, as previously described. In general, 
the selected representative pair is that pair having the most dis-similar constituent regions-of- 
interest, based on the determined similarity distances. Operation then continues to step 
SI 424. It should be appreciate that in a case where only a single pair of regions-of-interest 
has been defined, it is selected as the representative pair of regions-of-interest. Then, in step 
SI 424, operation returns to step S1430. 

[0111] Fig. 1 4 is a flowchart outlining one exemplary embodiment of a method for 
selecting valid edge points using the membership-image of Fig. 12 according to this 
invention. Beginning in step SI 650, operation proceeds to step SI 651 where a first or next 
edge point is selected. Then, in step S1652, a new type of a pair of regions of interest EROI1 
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and EROI2, generally unrelated in function and position to the previously defined regions-of- 
interest, is defined for the selected edge point. In one exemplary embodiment, EROI1 and 
EROI2 are 1 1 -by- 11 pixel squares, centered on the scan line corresponding to the selected 
edge point, and centered 10 pixels away from the selected edge point, on respective opposite 
sides. Operation then continues to step SI 653. 

[0112] In step SI 653, a determination is made of the degree of conformity of the 
membership image pixel values in the new pair of regions of interest EROI1 and EROI2. 
Operation then continues to step S1654. 

[0113] It should be appreciated that the membership image pixels have a range of 
possible values between a first value representing perfect membership in the class 
corresponding to RROI1, and a second value representing perfect membership in the class 
corresponding to RROI2. The pixels in each respective new region of interest, EROI1 and 
EROI2, should generally conform to their respective sides of the membership image 
boundary. In one exemplary embodiment, if a pixel value lies closer to the first value, it 
conforms to the class of RROI1 and if it lies closer to the second value, it conforms to the 
class of RROI2. In another exemplary embodiment, the membership image pixel values are 
compared to a threshold determined during a learn mode based on one or more determined 
edge point's membership values for evaluating membership conformity. 

[0114] In step SI 654, a determination is made whether the degree of membership 
conformity meets a predefined "good" criteria. That is, in step SI 654, the edge points in the 
initial set of edge points PEI are analyzed to determine whether a detected edge point should 
be discarded from the initial set of edge points as being an invalid edge point. The detected 
edge point is not discarded, for example, if a predetermined proportion of the pixels in EROI1 
conform to the criterion representing their side of the boundary (such as CV1, a property of 
RROI1, or the like) and a predetermined proportion of the pixels in EROI2 conform to the 
criterion representing their side of the boundary. If the "good" criteria is met, then operation 
jumps to step S1656. Otherwise, operation proceeds to step S1655, where the selected edge 
point is discarded from the set of initial edge points. Operation then proceeds to step SI 656. 
In one exemplary embodiment, the proportion of pixel conforming in each region EROI1 and 
EROI2 must be at least 85%, otherwise the selected edge point is discarded. It should be 
appreciated that low conformity corresponds to a noisy or anomalous region, which tends to 
indicate an invalid edge point. The predetermined proportion may be adjusted depending on 
the reliability desired for the "accepted" edge points. Furthermore, it should be appreciated 
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that different types of criteria for distinguishing one side of the boundary from the other may 
be used as the conformity criteria during run mode operations and training mode operations, 
respectively, depending on the data conveniently available in each of the two modes. 

[0115] In step SI 656, a determination is made whether there are any remaining edge 
points to be analyzed. If so, the operation returns to step SI 651. Otherwise, operation 
proceeds to step SI 657. 

[0116] In step S 1 657, one or more feature distance values D are determined 
corresponding to each remaining edge point that was not discarded in step S 1 655. In one 
exemplary embodiment, the Fisher distance between the previously described EROI1 and 
EROI2 corresponding to each remaining edge point is determined, based on all features 
images in the selected subset of feature images. In this case, a single distance value D results 
for each remaining edge point. Next, in step S1658, one or more corresponding difference 
parameters d are determined based on the one or more determined distance values D for the 
remaining edge points. The difference parameter(s) d may be at least temporarily stored as 
tool-related data. For example, the minimum of the Fisher distance values D, just described, 
may be determined as a single difference parameter d. Operation then continues to step 
S1659. 

[0117] In step S 1 659, a first or next edge point is selected from the remaining edge 
points PE of the set of initial edge points PEL Operation then continues to step SI 660. 

[01 18] In step S 1 660, a determination is made whether the one or more feature 
distances (D) for the selected edge point determined in step SI 657 is less than the 
corresponding one or more difference parameters (d) determined in step S1658. If the one or 
more feature distances (D) for the selected edge point are not less than the corresponding one 
or more difference parameters (d), then operation jumps to step SI 662. Otherwise, operation 
proceeds to step SI 661, where the selected edge point is discarded from the set of remaining 
edge points PE. Operation then continues to step SI 662. In step SI 662, a determination is 
made whether there are any remaining edge points to be validated. If so, then operation 
returns to step S1659. Otherwise, operation goes to step S1663, where operation returns to 
step SI 670. 

[0119] It should be appreciated that the difference parameters d determined by the 
operations of the step SI 65 7 can be saved and used in association with the associated trained 
edge tool during the run mode, in a manner similar to the applicable operations described 
with reference to the steps S1657-S1662. The effect is to tend to insure that the membership 
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image created at run time is at least approximately as suitable for edge-detection as the 
membership image used for training. It should be further appreciated that if d is set to the 
minimum value previously described, the steps S1659-S1662 need not be performed in a tool 
training mode. It should be further appreciated that the sets of operations approximately 
corresponding to the steps SI 651 -SI 656, and the steps S1657-S1662, respectively, both tend 
to insure the reliability of the remaining edge points, Thus, the screening method used in 
either set of operations can generally also be implemented alone. Although the reliability and 
accuracy may be somewhat affected for some edges, significant benefits will be retained in 
such embodiments of the systems and methods according to this invention. 

[0120] Fig. 1 5 is a flowchart outlining one exemplary embodiment of a method for 
detecting the location of a similar specific case of an edge in a different but similar specific 
case of an input image using the parameters defined according to the setup method outlined in 
Figs. 7-14 according to this invention. As previously discussed, the edge detection systems 
and methods, and more specifically, the boundary detection tool, have been set up by the 
operation previously discussed to detect specific edges within a specific input image using 
specific image-derived parameters. Accordingly, the edge detection systems and methods of 
this invention can now be used in an automated mode to locate edges or boundaries in a 
different but similar case of that input image during a run mode. Because the operation of the 
run mode according to the edge detection systems and methods of this invention encompasses 
many of the same steps as previously discussed in the set-up mode, a detailed description of 
steps S2100-S2400 and S2600-S2700 will be omitted because these steps are similar to the 
corresponding steps in Figs. 7-12 but some parameters previously determined and 
accepted/stored during "learn" mode, are used at "run mode." 

[0121] Beginning in step S2000, operation proceeds to step S2100, where a first or 
next image is acquired. Then, in step S2200, the area of interest and the one or more scan 
lines are determined using the parameters determined at "learn mode". Next, in step S2300, 
one or more feature images are generated based on the previously selected filters stored as 
tool-related data. Then, in step S2400, the membership-image is generated based on the set 
of feature images generated in the operations of the step 2300, and the previously-discussed 
classification vectors CV1 and CV2. Operation then proceeds to step S2500. 

[0122] It should be appreciated that in various other exemplary embodiments, the 
membership image may be generated based on various different combinations of retained 
tool-related data, and currently generated data. For example, in a first embodiment, the 
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classification vectors CV1 and CV2 are the vectors determined during the training or learn 
mode, and the membership image pixel values are determined accordingly. In a second 
embodiment, current classification vectors CV1 and CV2 are determined from the current set 
of feature images, using a pair of RROI's based on an RROI definition determined during the 
training or learn mode. In a third embodiment, current RROI1 and RROI2 are determined 
using the operations of the steps 1410-1420, current CV1 and CV2 are determined using the 
operations of the step 1450, and the membership image pixel values are determined 
accordingly. It should be appreciated that the second and third embodiments will be 
somewhat more time consuming than the first embodiment, but all three embodiments derive 
the benefits associated with using the previously selected filters stored as tool-related data. 
Various other combinations and alternative will be apparent to one skilled in the art. 

[0123] In step S2500, one or more edge points are detected in each scan line and the 
"good" edge points are selected. Because this operation is different from the edge point 
detection and selection process described with respect to step SI 600 of Figs. 7, 12 and 14, a 
more detailed description of this operation will be described with reference to Fig. 16. Next, 
in step S2600, the location of the edge to be detected along each scan line having a remaining 
edge point that has not been discarded in step S2500 is refined and the edge location is finally 
determined, all as previously described with reference to step SI 700 of Fig. 7. Then, in step 
S2700, a determination is made whether another input image is to be acquired. If so, then 
operation jumps back to step S2100. Otherwise, operation proceeds to step S2800, where the 
operation of the run mode method ends. 

[0124] Fig. 16 is a flowchart outlining in greater detail one exemplary embodiment 
of the method for selecting edge point locations of Fig. 15 according to this invention. 
Beginning in step S2500, operation proceeds to step S2510, where an initial set of edge points 
for the determined set of scan lines is detected. This set of edge points is based on the 
membership-image generated in step S2400. Next, in step S2520, an unselected edge point is 
selected. Then, in step S2530, the feature distance (D) for the selected edge point is 
determined from that edge point, as previously described with reference to step SI 657 of Fig. 
14. Then, operation proceeds to step S2540. 

[0125] In step S2540, a determination is made whether the one or more feature 
distances D for the selected edge point are less than the corresponding one or more difference 
values d previously defined in step S1658 in Fig. 14. If so, operation proceeds to step S2550. 
Otherwise, operation jumps to step S2560. In step S2550, because the one or more feature 
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distances D for the selected edge point are less than the corresponding one or more difference 
values d, the selected edge point is discarded from the initial set of edge points. Then, in step 
S2560, a determination is made whether there are any remaining unselected edge points. If 
so, then operation returns to step S2520. Otherwise, operation proceeds to step S2570, where 
operation returns to step S2600. It should be appreciated that in various exemplary 
embodiments, the operations described with reference to the steps SI 651 -SI 65 6 are 
performed prior to performing the step S2570 in run mode, to further increase the reliability 
of the remaining edge points. For example, the operations may be performed just after the 
step 2560, or just after the step 2510. 

[0126] The control portion 1 00 in various exemplary embodiments, is implemented 
on a programmed general purpose computer. However, the control portion 1 00 in accordance 
with this invention can also be implemented on a special purpose computer, a programmed 
microprocessor or microcontroller and peripheral integrated circuit elements, an ASIC or other 
integrated circuit, a digital signal processor, a hardwired electronic or logic circuit such as a 
discrete element circuit, a programmable logic device such as a PLD, PLA, FPGA or PAL, or 
the like. In general, any device, capable of implementing a finite state machine that is in turn 
capable of implementing the flowcharts shown in Figs. 7-15, can be used to implement control 
portion 100 in accordance with this invention. 

[0127] The memory 1 30 can be implemented using any appropriate combination of 
alterable, volatile or non-volatile memory or non-alterable, or fixed, memory. The alterable 
memory, whether volatile or non-volatile, can be implemented using any one or more of static 
or dynamic RAM, a floppy disk and disk drive, a writable or re-rewriteable optical disk and disk 
drive, a hard drive, flash memory or the like. Similarly, the non-alterable or fixed memory can 
be implemented using any one or more of ROM, PROM, EPROM, EEPROM, an optical ROM 
disk, such as a CD-ROM or DVD-ROM disk, and disk drive or the like. 

[0128] It should be understood that each of the circuits or other elements 1 50-1 80 
and 305-379 shown in Fig. 1 can be implemented as portions of a suitably programmed 
general purpose computer. Alternatively, each of the circuits or other elements 150-180 and 
305-379 shown in Figs. 1 can be implemented as physically distinct hardware circuits within 
an ASIC, or using a FPGA, a PDL, a PLA or a PAL, or using discrete logic elements or 
discrete circuit elements. The particular form each of the circuits or other elements 150-180 
and 305-379 shown in Fig. 1 will take is a design choice and will be obvious and predicable 
to those skilled in the art. 
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[0129] Moreover, the control portion 100 can be implemented as software executing 
on a programmed general purpose computer, a special purpose computer, a microprocessor or 
the like. The control portion 100 can also be implemented by physically incorporating it into 
a software and/or hardware system, such as the hardware and software systems of a vision 
system. 

[0130] While the invention has been described with reference to what are preferred 
embodiments thereof, it is to be understood that the invention is not limited to the preferred 
embodiments or constructions. To the contrary, the invention is intended to cover various 
modifications and equivalent arrangements. In addition, while the various elements of the 
preferred embodiments are shown in various combinations and configurations, which are 
exemplary, other combinations and configurations, including more, less or only a single 
element, are also within the spirit and scope of the invention. 



